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Abstract 

The  United  States  Navy  is  undergoing  a  rapid  transformation  in  the  operations  it  conducts  -  the 
types  of  enemies  it  faces,  the  resources  it  has  to  draw  upon,  the  capabilities  it  can  deliver,  the 
manner  in  which  it  coordinates  with  other  branches  of  the  armed  services,  and  the  organizational 
stmctures  it  uses  to  bring  those  new  resources  and  capabilities  to  bear  against  a  new  generation  of 
enemies.  To  accommodate  this  rapid  transformation,  a  revolution  has  been  occurring  that  began 
with  the  development  of  the  concept  of  “network-centric  warfare”  (NCW).  NCW  promises  to 
deliver  unprecedented  operational  tempo  and  situational  awareness  through  networked 
connectivity.  For  the  Navy,  the  NCW  concept  has  evolved  into  the  definition  of  FORCEnet  as  a 
future  organizing  principle.  Given  this  rapid  transformation,  several  questions  emerge  regarding 
how  best  to  realize  the  FORCEnet  vision.  These  questions  involve  issues  such  as  organizational 
design,  information  flow,  information  filtering,  and  display  technologies.  Accordingly,  in  this 
report,  we  describe  an  effort  to  develop  an  integrated  testbed  to  explore  FORCEnet  concepts  and 
technologies.  The  testbed  is  unique  in  that  it  serves  to  unite  research  on  novel  FORCEnet 
architectures  with  research  designed  to  develop  innovative  information  displays  to  support 
network-centric  operations.  Our  intent  in  this  report  is  to  briefly  describe  this  testbed,  which  will 
enable  future  experimentation  and  validation  of  emerging  concepts. 


The  research  reported  here  was  sponsored  by  the  Office  of  Naval  Research,  Contract  No.  N00014-02-C- 
0233,  under  the  direction  of  Gerald  Malecki. 
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Introduction 


The  United  States  Navy  is  undergoing  a  rapid  transformation  in  the  operations  it  conducts  -  the 
types  of  enemies  it  faces,  the  resources  it  has  to  draw  upon,  the  capabilities  it  can  deliver,  the 
manner  in  which  it  coordinates  with  other  branches  of  the  armed  services,  and  the  organizational 
structures  it  uses  to  bring  those  new  resources  and  capabilities  to  bear  against  a  new  generation  of 
enemies.  To  accommodate  this  rapid  transformation,  a  revolution  has  been  occurring  that  began 
with  the  development  of  the  concept  of  “network-centric  warfare”  (NCW).  NCW  promises  to 
deliver  unprecedented  operational  tempo  and  situational  awareness  through  networked 
connectivity.  For  the  Navy,  the  NCW  concept  has  evolved  into  the  definition  of  FORCEnet  as  a 
future  organizing  principle.  FORCEnet  is  viewed  as  the  operational  construct  and  architectural 
framework  for  naval  warfare  in  the  information  age,  integrating  warriors,  sensors,  command  and 
control,  platforms,  and  weapons  into  a  networked,  distributed  combat  force.  It  is  envisioned  that 
FORCEnet  will  provide  the  architecture  to  increase  combat  capabilities  through  aligned  and 
integrated  systems,  functions,  and  missions.  Like  NCW  in  general,  it  promises  to  improve 
situational  awareness,  accelerate  speed  of  decision  making,  and  greatly  distribute  combat  power. 

Given  this  rapid  transformation,  several  questions  emerge  regarding  how  best  to  realize  the 
FORCEnet  vision.  These  questions  involve  issues  such  as  organizational  design,  information 
flow,  information  filtering,  and  display  technologies.  Accordingly,  in  this  report,  we  describe  an 
effort  to  develop  an  integrated  testbed  to  explore  FORCEnet  concepts  and  technologies.  The 
testbed  is  unique  in  that  it  serves  to  unite  research  on  novel  FORCEnet  architectures  with 
research  designed  to  develop  innovative  information  displays  to  support  network-centric 
operations.  Our  intent  in  this  report  is  to  briefly  describe  this  testbed,  which  will  enable  future 
experimentation  and  validation  of  emerging  concepts. 

We  introduce  the  testbed  below  by  briefly  addressing  the  Adaptive  Architectures  for  Command 
and  Control  and  the  Command21  programs,  which  serve  as  the  basis  for  the  testbed  and  planned 
research.  We  then  describe  the  testbed  and  present  initial  results  from  a  recent  demonstration  of 
its  capabilities. 

The  A2C2  Program 

The  Adaptive  Architectures  for  Command  and  Control  (A2C2)  program  is  ideally  positioned  with 
theories,  methods,  findings,  and  tools  that  can  directly  support  the  study  and  evaluation  of  this 
rapid  change  in  naval  operations.  The  A2C2  program  is  a  collaborative  effort,  coordinated  by 
Aptima,  Inc.,  and  sponsored  by  the  Office  of  Naval  Research.  The  program  embraces  multiple 
partners  including  the  U.S.  Naval  War  College,  Naval  Post  Graduate  School,  University  of 
Connecticut,  Carnegie  Mellon  University,  George  Mason  University,  and  Michigan  State 
University.  The  objective  of  the  A2C2  program  has  been  to  explore  novel  command  and  control 
organizational  designs  that  support  emerging  FORCEnet  concepts.  Indeed,  from  the  beginning, 
A2C2  was  envisioned  as  a  program  to  develop  organizational  structures  that  would  be  adaptable, 
possessing  the  ability  to  rapidly  change  to  meet  the  demands  presented  by  new  enemies  and  new 
missions.  Hence,  A2C2  is  now  in  a  unique  position  to  advance  the  military’s  understanding  of 
how  its  C^  organizational  structures  can  evolve  in  new  directions  to  take  advantage  of  network 
connectivity  to  conduct  new  types  of  operations.  In  particular,  over  the  last  several  years,  the 
A2C2  project  has  explored  many  new  organizational  structures,  concepts,  tools,  and  displays  to 
aid  the  warfighter  in  organizational  adaptation  and  effectiveness.  Two  components  of  the  testbed 
are  highlighted  below  that  were  developed  as  part  of  the  A2C2  program:  the  A2C2  Distributed 
Dynamic  Decision-making  Simulation  and  the  A2C2  Decision  Support  System. 
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The  A2C2  Distributed  Dynamic  Decision-making  Simulation 

The  cornerstone  of  the  A2C2  program  is  model-based  experimentation,  in  which  models  of 
organizations  are  utilized  to  define  operational  concepts,  hypotheses,  and  scenarios  that  can  be 
evaluated  through  experimentation  involving  human  participants  (see  Kleinman  et  ah,  2003). 
Experimental  study  of  these  organizations  has  been  conducted  using  the  Distributed  Dynamic 
Decision-making  (DDD)  simulation.  The  DDD  application  is  a  versatile  distributed  multi-person 
simulation  and  software  tool  for  understanding  command  and  control  issues  in  a  dynamic  team 
environment  (Serfaty  &  Kleinman,  1985;  Kleinman  &  Serfaty,  1989).  The  application  offers 
flexibility  in  that  it  provides  the  ability  to  simulate  different  domains  and  scenarios  to  study 
realistic  and  complex  military  team  decision-making.  The  DDD  team-in-the-loop  simulation 
environment  has  already  been  proven  as  an  effective  test-bed  for  conducting  experiments  in 
adaptive  architectures  for  Joint  Task  Force  (JTF)  missions  (e.g.,  Diedrich  et  al.,  2003;  Entin  et  al., 
2003).  Figure  1  shows  a  full-screen  snapshot  of  the  DDD  running  the  JTF  scenario  used  as  a 
basis  for  Diedrich  et  al.  (2003)  and  Weil  et  al.  (2004)  to  study  aspects  of  organizational 
adaptation.  In  general,  the  DDD  provides  an  extensive  set  of  capabilities  for  supporting 
experiments  of  teamwork  in  the  military.  The  current  generation  of  the  DDD  is  a  distributed  real¬ 
time  simulation  environment 
implementing  a  complex  synthetic 
team  task  that  includes  many  of  the 
behaviors  at  the  core  of  almost  any 
multi-person  mission:  assessing  the 
situation,  planning  response  actions, 
gathering  information,  sharing 
information,  allocating  resources  to 
accomplish  tasks,  coordinating 
actions,  and  sharing  or  transferring 
resources. 


The  A2C2  Decision  Support 
System 

As  a  key  part  of  the  A2C2  program, 
and  critical  to  the  system  outlined 
here,  the  University  of  Connecticut  Figure  1.  DDD  Simulation, 

and  the  Naval  Postgraduate  School  have  been  working  on  the  development  of  a  Decision  Support 
System  (DSS)  (Meirina  et  al.,  2004)  to  facilitate  decision-making  and  adaptation  in  the 
organizational  structures  studied  in  the  A2C2  project.  In  essence,  the  DSS  aims  to  enhance 
organizational  awareness  and  decision  making  by  providing  real-time  information  relevant  to  the 
organizational  structure,  organizational  adaptation,  organization  process,  and  environmental 
attributes  of  the  battle-space  and  mission.  The  DSS  is  envisioned  as  a  prototype  for  supporting 
organizational  adaptation  in  network-centric  operations. 

The  DSS  tool  incorporates  synthetic  agents  that  provide  interactive  “what-if  ’  analyses  to  provide 
richer  information  content  and  reduce  cognitive  load.  The  agent-driven  DSS  prototype  provides 
four  display  areas  as  follows: 

1.  Time  Window:  contains  real-time  task-related  information,  which  is  customized  for  each 
specific  decision  maker  (DM).  See  Figure  2.i 
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-  Time-window  chart:  provides  real-time  data  updates  for  tasks,  whose  resource 
requirements  and  attributes  match  those  of  the  specific  decision  maker,  i.e.,  the 
owner  of  the  Time-Window  display.  The  colors  of  the  displayed  task-bars  indicate 
the  status  of  the  tasks,  i.e.,  green  -  to  be  processed,  blue  -  being  processed,  red  - 
missed,  and  grey  -  completed.  Shaded  task-bars  suggest  DM-DM  coordination 
requirements. 

-  Task  information:  provides  detailed  information  for  a  selected  task,  which  includes 
resource  requirements  for  task  execution,  attributes,  and  an  estimate  of  available  time 
left  to  process  the  task. 

-  Coordination  requirements:  indicates  coordination  requirements  among  listed 
decision  makers  to  process  a  selected  task. 

2.  Asset  Status:  contains  real-time  information  on  the  status  and  availability  of  assets.  See 
Figure  2.ii. 

-  Asset  availability:  the  rows  hold  the  list  of  the  platforms  (e.g.,  ships)  and  the  columns 
show  the  list  of  the  sub-platforms  (e.g..  Aircraft).  The  platforms  -  sub-platforms  table 
indicates  the  number  of  available  sub-platforms  from  the  number  of  available 
platforms. 

-  Sub-platform  information:  provides  detailed  information  on  sub-platforms,  viewable 
by  platform,  sub-platform,  or  DM-owner.  The  information  includes  the  platform 
locations,  availability  status,  estimates  of  waiting  periods,  and  ownerships. 

3.  Decision  Maker  Workload  Chart:  illustrates  organizational  process  measures.  See 
Figure.  2.  iii.  This  type  of  displays  allows  the  decision  makers  to  asses  their  individual 
decision  making  in  comparison  to  that  of  others  to  facilitate  organizational  adaptation. 
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Figure  2.  The  Decision  Support  System  for  the  DDD 
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4.  Performance  Summary:  indicates  the  overall  assessment  of  organizational  performance. 

See  Figure  2.iv.  This  display  illustrates  another  trigger  for  organizational  adaptation. 

The  agent-driven  DSS  prototype  is  designed  to  provide  critical  information  to  the  decision 
makers,  in  real  time,  thereby  facilitating  more  effective  decision-making  and  collaboration 
through  improved  situational  awareness.  Figure  2  provides  screen  shots  of  the  display  windows 
depicting  the  four  DSS  display  areas.  This  prototype  provides  a  starting  point  for  research  on 
how  best  to  display  these  types  of  information  to  facilitate  adaptation  and  improved  team 
performance. 

The  Commandll  Program 

In  addition  to  the  DDD  and  DSS,  the  project  outlined  in  this  report  sought  to  integrate  the 
Commandll  Research  Program  (Feher  et  al.,  2003;  Smallman  et  ah,  2001).  The  Commandll 
project  offers  a  platform  for  integration  of  displays  designed  to  support  NCW  with  the  AlCl 
research  program.  The  Office  of  Naval  Research  sponsored  the  Commandll  project,  managed  by 
SPAWAR  Systems  Center,  San  Diego  and  supported  by  Pacific  Science  &  Engineering  and 
SPAWAR  has  developed  an  innovative  concept  known  as  the  Knowledge  Web  (K-Web)  and 
supporting  technologies  such  as  the  K-Wall  and  K-Desk.  K-Desks  were  used  as  part  of  the 
integrated  testbed  discussed  here.  The  K-Desk  is  a  six  display  system  that  utilizes  web-based 
technologies  to  integrate  mission  relevant  information,  tools,  and  displays  to  facilitate  group 
interaction  and  augment  decision-making  (Pester-DeWan  et  al.,  2003).  Mission  relevant 
information  is  graphically  displayed  in  easily  viewable  HTML  formats  thought  to  support 
decision  making.  The  type  of  information  displayed  in  each  of  the  displays  of  the  K-Desk  can  be 
chosen  by  the  commander  from  a  number  of  available  displays  for  maximum  flexibility  and 
customizability.  In  this  way  the  tool  facilitates  the  utilization  of  the  myriad  of  information 
available  under  emerging  network-centric  operations.  A  picture  of  the  K-Desk  is  provided  in 
Figure  3.  Critical  for  the  needs  outlined  here,  the  K-Desk  enables  the  study  of  emerging 
FORCEnet  concept  via  displays  designed  to  support  the  next  generation  of  information  sharing. 
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Knowledge  Web  (K-Web):  The  A2C2  and  Command21  Integrated  Testbed 


By  bringing  these  three  components  together,  our  objective  in  the  work  reported  here  was  to 
develop  an  integrated  testbed  with  sufficient  operational  fidelity  to  study  FORCEnet  concepts  and 
tools  in  a  laboratory  setting.  The  A2C2  project  made  available  novel  organizational  structures, 
decision-support  tools  (i.e.,  DSS),  and  the  DDD  team  simulation.  The  Commandll  project 
provided  an  operationally  relevant  platform  (i.e.,  K-Desk)  for  integration  and  dissemination  of 
shared  mission-relevant  knowledge  and  tools  that  are  customizable  based  upon  the  needs  of  the 
individual  commander.  The  result  of  the  integration  is  the  framework  depicted  at  a  higher  level 
in  Figure  4.  The  next  section  identifies  the  technical  aspects  of  integrating  of  the  DDD,  DSS,  and 
K-Desk. 


Figure  4.  Diagram  of  K-Web  integration 

Technical  Aspects  of  Integrating  the  DDD,  DSS,  and  K-Desk 

In  order  to  create  an  integrated  system  that  incorporated  the  DDD,  DSS  and  K-Desk,  it  was 
necessary  to  develop  a  new  software  infrastructure  that  would  allow  for  data  to  be  accessible 
from  the  DDD  by  the  DSS  in  real-time.  This  was  a  challenge  given  that  these  two  systems  work 
under  different  operating  systems:  the  DDD  operates  under  Linux,  while  the  DSS  is  a  Windows 
application.  In  addition,  the  DSS  utilized  an  existing  database  (DB)  that  would  have  to  be 
populated  in  real-time  in  order  for  the  DSS  to  function  properly. 

To  address  these  challenges,  we  developed  new  functionality  that  allowed  the  DDD  to 
communicate  with  windows-based  client  applications  through  an  existing  mechanism  called  the 
DDD  External  Conduit  (DEC).  With  this  infrastructure  in  place,  we  then  developed  a  new  client 
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application,  called  the  DDD  State  Module,  that  modeled  the  DDD  dynamic  state  information 
based  on  messages  received  from  the  DDD  (via  the  DEC)  that  could  in  turn  share  those  data  with 
other  windows-based  clients.  We  then  developed  another  client  application,  called  the  DB 
Module  that  could  take  this  dynamic  state  information  and  populate  the  DSS  database.  These 
different  integrated  capabilities  are  shown  in  Figure  5. 


Figure  5.  DDD  Interface  and  Support  Modules  Design 


The  DSS  synthetic  agents  would  then  use  those  data  to  generate  and  update  the  four  information 
displays  described  above.  Integration  of  the  DSS  information  displays  within  the  K-Desk  was 
straightforward  given  their  implementation  as  web-enabled  Active  Server  Pages. 

Based  on  this  design,  we  sought  to  demonstrate  the  feasibility  of  this  testbed  as  well  as  assess 
user  acceptance  and  obtain  initial  feedback.  We  therefore  conducted  a  demonstration  of  the 
system  in  August  of  2004  at  the  U.S.  Naval  War  College. 

Method 

Participants 

Twelve  officers  served  as  participants  in  the  demonstration.  All  participants  were  Lieutenant 
Commanders  or  higher  and  were  students  at  the  U.S.  Naval  War  College  in  Newport,  RI.  The 
twelve  officers  were  organized  into  two  teams  of  six  individuals  each  for  two  separate  data 
collection  sessions. 

Simulation  Environment 

From  the  perspective  of  the  participants,  the  simulation  environment  was  built  on  the  integrated 
test  bed  or  Knowledge  Web  (K-Web).  For  the  purpose  of  the  demonstration,  the  K-Web  was  the 
integration  of  the  K-Desk,  the  DDD,  static  mission-relevant  information  (e.g.,  mission  plan,  etc.), 
and  the  DSS.  Each  of  the  six  K-Desks  (one  for  each  team  member)  consisted  of  six  video 
displays,  two  mice,  and  supporting  software  implemented  in  a  single  testing  room  at  the  U.S. 
Naval  War  College.  The  DDD  simulation  was  presented  on  a  central  display  of  the  Knowledge 
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DDD 


Desk.  The  remaining  five  displays  were  used  to  display  mission  relevant  information  and  the 
graphical  information  provided  by  the  DSS.  The  DSS  provided  dynamic  information  about  team 
and  individual  performance.  One  mouse  manipulated  the  static  and  dynamic  information  displays 
and  one  was  used  by  the  DDD  simulation. 

Organizational  Structure  and  Mission  Manipulation 

For  the  purpose  of  this  demonstration,  the  methodology  described  by  Diedrich  et  al.  (2003)  was 
followed  so  that  the  efficacy  of  the  integrated  testbed  could  be  assessed  using  a  well-understood 
scenario  and  organizational  structure.  In  this  case,  a  single  organizational  structure  (Functional 
organization,  F)  was  employed,  which  was  characterized  by  the  types  of  assets  within  each  team 
member’s  control.  In  this  demonstration,  each  participant  was  given  control  of  functionally 
similar  assets  that  were  dispersed  throughout  the  mission  battle  space;  for  example,  they  might  be 
given  all  the  Intelligence-Surveillance-Reconnaissance  or  all  the  Strike  assets. 

Using  this  structure,  each  team  participated  in  two  data-collection  sessions.  Scenarios  were 
developed  that  were  either  well  suited  to  the  F  organizational  structure  (functional  scenario,  f)  or 
not  well  suited  to  the  F  organizational  structure  (divisional  scenario,  d).  A  mission  was 
considered  to  be  congruent  if  the  team’s  organizational  structure  was  matched  with  its  associated 
scenario  (i.e.,  Ff).  In  contrast,  a  session  in  which  the  team  and  the  scenario  structure  were 
mismatched  was  considered  incongruent  (i.e.,  Fd).  Congruence  was  achieved  by  defining  task 
requirements  within  a  scenario  that  matched  the  asset  capabilities  of  the  organizational  structure 
to  varying  degrees,  thus  reducing  (Ff)  or  increasing  (Fd)  the  need  for  coordination  among  players; 
see  Diedrich  et  al.  (2003)  and  Entin  et  al.  (2005)  for  complete  details. 

Procedure 

Participants  were  briefed  on  the  purpose  of  the  demonstration  and  demographic  information  was 
collected.  Participants  then  received  training  on  the  battlespace  simulation  generated  by  the 
DDD.  They  received  DDD  “buttonology”  training  (i.e.,  how  to  use  the  simulation)  followed  by 
training  designed  to  provide  the  skills  necessary  to  perform  the  scenarios  in  a  team  environment. 
Training  on  the  displays  of  the  K-Web  occurred  independently  of  DDD  training.  Instruction  on 
the  scenario-specific  background  information  was  presented  as  information  related  to  the  mission 
(e.g.,  road  to  war,  commander’s  intent,  tasking  order,  etc.).  Before  the  first  scenario,  participants 
received  a  briefing  on  the  DSS  and  the  importance  of  the  information  it  provided. 

The  first  scenario  was  designed  to  be  congruent  with  the  organizational  structure  the  team  had 
been  assigned  (Ff  condition).  The  mission  was  to  prepare  the  battlespace  for  the  insertion  of 
follow-on  forces  using  land,  sea,  and  air  assets.  There  were  several  objectives,  including  the 
destruction  or  capture  of  a  command  center,  two  air  bases,  two  naval  bases,  and  a  final  port.  In 
addition  to  fixed  objectives  there  were  targets  of  opportunity,  such  as  missile  launchers  that  had 
be  identified  and  engaged  before  they  fired.  Communication  among  team  members  was 
emphasized  as  well  as  the  use  of  communication  discipline  (e.g.,  the  use  of  call  signs  and 
brevity).  In  contrast  to  previous  work  (e.g.,  Diedrich  et  al.,  2003),  all  scenarios  were  conducted  at 
half  normal  speed  to  facilitate  use  of  the  DSS  and  K-Desk  information. 

At  the  completion  of  the  first  scenario,  an  after  action  review  (AAR)  was  conducted  to  discuss 
team  performance  and  lessons  learned.  An  intelligence  brief  was  then  presented  to  the 
participants  to  explain  that  the  enemy  had  changed  their  tactics  based  upon  the  strategies 
employed  by  Blue  forces  during  the  first  scenario.  The  mission  task  requirements  had  changed 
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resulting  in  the  need  for  new  coordination  and/or  collaboration  strategies  to  prosecute  tasks.  This 
change  in  mission  task  requirements  represented  a  shift  in  the  design  of  the  scenario  making  the 
second  scenario  incongruent  with  the  organizational  structure  the  teams  were  using,  producing 
the  Fd  condition.  A  small  strategy  session  occurred  based  upon  the  Intel  brief  and  task  graphs  to 
allow  the  team  to  plan  task  prosecution  during  the  second  scenario.  At  the  end  of  this  scenario  a 
second  AAR  was  conducted  to  discuss  outcome  and  attitudes. 

Measures 

The  measurement  focus  for  the  demonstration  was  on  feasibility  of  integration,  user  acceptance, 
and  user  perceived  value.  The  feasibility  of  integration  examined  the  integration  of  software 
components  and  the  capabilities  of  those  individual  components  to  meet  the  challenges  presented 
by  the  experiment  tempo,  in  particular  the  database  capabilities  and  communication  between  the 
components.  User  acceptance  and  perceived  value  were  assessed  both  during  the  demonstration 
through  observation  of  screen  use  and  at  the  completion  of  the  experiment  using  an  AAR.  Some 
of  the  talking  points  used  during  this  AAR  included: 

•  Overall  benefit,  what  impact  did  this  set  of  tools  have  on: 

o  Decision  making:  To  what  extent,  if  any  benefit  will  this  group  of  technology 
have  on  your  ability  to  make  decisions 

o  Workload 

o  Ability  to  improve  situational  awareness 

o  Value  of  concurrent  viewing:  benefit  of  being  able  to  see  multiple  sources/views 
of  information  concurrently 

o  Information  utility:  what  information  was  the  most  useful  from  all  the  systems  as 
well  as  displays  that  were  useful  or  were  not  useful. 

•  Recommendations 


Results 


Successes 

The  demonstration  was  sueeessful  given  the  challenges  faced;  namely  the  integration  of 
complex  software  into  a  feasible  and  operationally  relevant  test  environment.  Critically, 
the  overall  system  performed  well,  the  system  was  stable,  and  user  acceptance  was  high. 
Participants  felt  that  the  tools  facilitated  decision-making  by  providing  information  that 
improved  their  ability  to  coordinate  the  plarming  and  distribution  of  assets.  Participants 
referred  to  information  regarding  resource  requirements,  asset  ownership,  and  task 
precedence  so  that  they  could  “keep  straight  who  owned  whaf’  as  well  as  know  what 
assets  were  available  to  coordinate  the  prosecution  of  tasks.  Participants  used  the 
information  to  prioritize  placement  of  assets  as  well  as  ensure  that  assets  were  available 
when  required. 

Additionally,  participants  felt  they  had  increased  situational  awareness,  particularly 
through  the  use  of  the  decision  support  tools  that  provided  performance  feedback  as  well 
as  mission  status  information.  Only  when  they  had  to  zoom  in  to  prosecute  a  task  did 
they  feel  they  were  at  risk  of  loosing  situational  awareness.  The  process  of  “zooming  in” 
is  a  common  practice  within  the  DDD  when  selecting  a  desired  task  for  processing. 
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Zooming  in  is  often  a  required  step  by  the  user  due  to  other  assets  or  tasks  being  in  elose 
proximity  to  the  desired  task  making  the  seleetion  of  that  task  diffieult. 

Observations 

Observations  of  team  member’s  information  use  indicated  that  a  majority  of  team  members  were 
using  both  decision  support  and  mission-relevant  information  to  facilitate  task  execution.  Team 
members  also  changed  the  type  of  information  displayed  as  circumstances  necessitated.  In 
addition,  many  would  leave  a  screen  either  blank  (information  minimized)  or  have  the  main  menu 
displayed  if  they  were  not  using  the  display.  The  screens  may  also  have  been  made  blank  to  limit 
distraction.  This  type  of  use  and  non-use  indicated  that  the  individuals  were  utilizing  the  quantity 
of  screens  to  their  advantage.  Observational  data  also  indicated  a  preference  for  weapons/assets 
status  display.  Participants  presumably  displayed  mission-relevant  information  in  order  to 
facilitate  task  and  mission  execution.  In  addition  to  the  weapons/assets  status  display,  one  team 
displayed  the  performance  summary  screen  almost  as  frequently,  perhaps  in  order  to  maintain 
situational  awareness  of  what  tasks  had  been  completed  and  what  yet  remained  to  be  prosecuted. 
Almost  all  of  the  different  display  types  were  used  at  one  time  or  another,  with  preferences 
apparently  driven  by  the  type  of  position  the  participant  played. 

Workload  varied  somewhat  for  the  different  positions,  but  participants  reported  that  they  felt  the 
integrated  system  reduced  workload  by  making  information  more  accessible.  This  accessibility 
helped  individuals  anticipate  increases  in  workload  due  to  changes  in  the  simulated  battlespace  so 
that  they  could  have  assets  ready  and/or  in  place  to  deal  with  upcoming  events.  Workload 
information  was  available  to  all  of  the  participants  in  many  forms  (e.g.;  taskload,  asset 
availability,  timeline).  In  addition,  this  provided  team  members  with  increased  awareness  of 
fellow  team  members’  workload,  reportedly  allowing  more  effective  coordination. 

Participants  recognized  the  benefit  of  the  integrated  suite  of  network-centric  tools  and  utilized 
these  tools  to  effectively  prosecute  tasks.  They  used  their  ability  to  view  a  variety  of  information 
concurrently  to  maintain  situational  awareness  and  make  more  appropriate  decisions  through 
more  appropriate  coordination.  Team  members  stated  that  they  liked  the  idea  of  being  able  to 
display  information  concurrently.  When  asked  about  the  number  of  displays  the  majority  of 
participants  felt  that  the  quantity  of  displays  was  reasonable  as  it  afforded  flexibility.  Some  team 
members  felt  that  the  number  of  display  screens  was  a  little  too  high.  Those  team  members  who 
felt  that  the  number  of  display  screens  was  high  agreed  that  the  ideal  number  of  displays  would 
be  four. 

Some  Challenges 

A  majority  of  participants  felt  that  a  lot  of  information  was  displayed  and  there  was  only  a  limited 
amount  of  time  available  to  digest  it,  and  that  they  needed  to  spend  a  fair  amount  of  time  learning 
the  types  of  information  available.  These  comments  highlight  a  common  problem  in  network¬ 
centric  warfare  -  a  tendency  toward  information  overload.  Information  management  training 
would  afford  some  help  as  would  intelligent  filtering  and  targeting  of  incoming  information.  As 
is  readily  apparent,  these  issues  reflect  a  primary  research  need  to  support  FORCEnet. 

Many  participants  expressed  a  preference  for  the  timeline  and  precedence  graph  displays  (the 
display  that  shows  what  tasks  must  be  completed  before  other  tasks  can  be  processed).  They  felt 
that  this  display  helped  in  maintaining  good  SA  and  helped  planning  for  future  demands. 
Although  the  weapons/assets  status  was  used  frequently,  many  participants  stated  they  would 
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have  liked  to  see  the  graph  provide  information  on  ammunition  availability,  as  well.  Another 
position  voiced  by  some  was  that  the  DSS  displays  were  overly  complex;  by  the  time  they  figured 
out  what  information  the  display  was  conveying,  they  could  have  derived  the  same  information 
from  direct  observation  of  the  ongoing  tactical  situation.  This  indicates  that  the  usability  of  the 
DDS  displays  must  be  improved  and  that  training  to  use  and  understand  the  DSS  displays  also 
needs  improvement. 

Discussions  indicated  that  several  participants  felt  the  necessity  for  two  displays  for  the  DDD 
simulation;  one  display  to  zoom  in  on  a  particular  task  to  be  prosecuted  and  one  display  to 
maintain  a  global  view.  When  it  was  pointed  out  that  the  large  screen  at  the  front  of  the  room 
displayed  a  global  view,  participants  noted  that  a  display  at  their  consul  would  be  easier  to  use.  A 
technology  that  the  participants  felt  was  missing  was  the  opportunity  to  use  chat  or  instant 
messenger.  It  was  noted  that  aboard  ship  chat  is  used  frequently  and  often  individuals  will  have 
more  than  one  conversation  going  at  a  time.  To  many  participants  the  use  of  verbal 
communications  is  reserved  for  action  items  and  not  for  general  communications. 

Conclusion 

In  summary,  these  demonstration  results  indicate  that  the  integration  process  was  a  success  and 
that  a  viable  testbed  to  investigate  FORCEnet  concepts,  processes,  and  technologies  is  at  hand.  In 
particular,  the  testbed  showed  the  potential  of  providing  war  fighters  with  an  integrated 
information-rich  environment  to  support  mission  execution,  and  as  such,  a  glimpse  into  the 
promise  of  NCW.  The  integration  of  the  agent  based  DSS  and  K-Desk  was  successful  and 
participants  readily  embraced  the  available  information  and  technology.  Moreover,  the 
participants  felt  the  available  information  and  technology  facilitated  the  completion  of  tasks  and 
the  overall  mission.  Based  on  these  results,  we  plan  on  using  this  testbed  to  further  explore 
emerging  FORCEnet  technologies  and  concepts. 

Several  improvements  are  planned  which  will  increase  the  effectiveness  of  this  testbed.  These 
include  increasing  the  response  time  of  the  decision  support  graphics  to  strengthen  the  association 
between  the  simulation  and  DSS,  as  outlined  in  the  results  section.  Interface  improvements,  such 
as  a  more  intuitive  ‘zoom’  feature  and  operationally  relevant  iconography  will  improve  simulated 
mission  performance.  A  deeper  integration  of  the  DDD  with  the  K-desk  and  DSS  would  increase 
the  types  of  information  that  can  be  presented  to  the  user.  Finally,  we  are  in  the  formative  stages 
of  developing  computer-based  agents  that  can  serve  as  adjuncts  to  the  human  participants  in  the 
DDD  simulation.  This  will  allow  the  investigation  of  organizational  structure  and  adaptation  in 
larger  organizations,  with  dozens  of  team  members. 

These  improvements  will  allow  us  to  examine  organizational  designs  in  modem  military 
organizations  such  as  the  Expeditionary  Strike  Group  (ESG),  based  on  emerging  FORCEnet 
concepts.  The  testbed  being  developed  will  allow  researches  to  better  understand  the  implication 
of  various  organizational  stmctures,  and  could  potentially  allow  research-based  recommendations 
for  novel  organizational  designs. 
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coordination  with  another  DM 
SOLID  BAR:  Task  can  be  done 
alone,  with  the  assets  DM  owns 
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(i)  Time-Window 


(ii)  Asset  Status 


(iii)  Decision-maker  Workioad 


(iv)  Performance  Summary 


space  Sc  mission 


Developed  by  the  University  of  Connecticut  &  NPS 
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Knowledge  Desk  Display  System 


K-Desk  is  web 
based 

Developed  to 

integrate  mission 
relevant  information, 
tools,  &  displays 

To 

Facilitate  group 
interaction  & 
augment  decision 
making 
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Information  displayed  in  each  of  the  screens  can  be  chosen  by  the  commander 
from  a  number  of  availabie  displays 

Facilitates  the  utilization  of  the  myriad  of  information  available  under  emerging 
network-centric  operations  (Developed  jointly  by  SPAWAR  &  PSE) 


Aptirna® 

^  Kdaia  '  C«rvrArid  EAgiAitnni 

DDD  Interface  &  Support  Module  Design 


Allowed  for  the  Linux 
based  DDD  to  communicate 
with  windows-based  DSS 

Employed  the  DDD  external 
conduit 

Two  new  application  were 
developed: 


DDD  state  module  -  modeled 

•A  new  software  infrastructure  was  the  DDD  dynamic  state  info 

developed  L messages  received 

irom  the  DDD  -  shared  with 
DSS 


•To  creat^g^g rated  system  that 
incorporated  the  DDD,  DSS,  & 

K-  Desk  DDD 


DB  module  -  takes  dynamic 
Globaitate  info  &  populates  the 
DSS  database 


Methodology 
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Participants 


Simuiation 

environment 


K-Desk 

testbed 


Testbed 

configuration 


12  Naval  Officers  -  Lt  Cmd  or  higher 
organized  into  two  teams  of  6 


Integrated:  DDD,  DSS,  &  K-Desk 
(testbed) 


Six  video  displays,  two  mice, 
&  supporting  software 


Middle  display  DDD  Sim,  other  five 
displays  used  to  display  mission 
relevant  info  &  graphic  info  provided  by 
DSS 
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Organizational  Structure  and  Mission  Scenario 

*  Scenarios  were  from  Diedrich  et  al.(2003) 

■  Teams  performed  under  a  Functional  organizational  structure 

■  Performed  a  congruent  (functional)  mission  followed  by  incongruent 
(divisional)  mission 

■  Functional  organization  characterized  by  the  type  of  assets  within  each 
members  control 

■  Controlled  functionally  similar  assets  that  were  dispersed 
throughout  the  mission  battle  space  (e.g.,  all  ISR  assets  or  all  Strike 
assets) 

■  Congruent  meant  team’s  organizational  structure  matched  mission 

■  Incongruent  implied  a  mismatch  between  organizational  structure  & 
mission 


Organization 


Mission 


Congruent 


Incongruent 
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Procedure 


■ 

I 

■ 


I 


DDD  training 

K-Desk  DSS  training 

Two  data  collection  scenarios:  congruent 
followed  by  incongruent  mission  scenario 


•  AAR  conducted  after  first  scenario  -  discussed  team 
performance  &  lessons  learned 

►  Intel  brief  then  informed  team  enemy  had  changed 
their  tactics 


■  Team  organizational  structure  now  incongruent  to  mission 
scenario 

■  AAR  conducted  at  end  of  second  scenario 

■  All  measures  observational  based  on  second  AAR 


DDD  scenarios  conducted  at  half  usual  speed  to 
facilitate  use  of  the  DSS  &  K-Desk  information 


Observations  (1) 
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*  Primary  goal  attained:  demonstration  was 
successful  C> 

•  System  performed  well  c> 

•  System  was  stable  €> 

•  User  acceptance  was  high  ^ 

*  Participants  felt 

•  Tools  facilitated  decision-making  by  providing  info 
that  improved  ability  to  coordinate  the  pianning  & 
distribution  of  assets 

■  Info  regarding  resource  requirements,  asset  ownership,  task 
precedence 

■  Helped  keep  straight  who  owned  what  &  what  assets  were 
available 

•  Increased  SA 


Observations  (2) 
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-  Team  members  used  both  DSS  and  mission-relevant 
information  (K-Desk) 

►  Changed  the  type  of  info  displayed  as  circumstances 
necessitated 

■  Many  left  a  screen  blank  to  limit  distractions 

►  Preference  for  weapons/assets  status  display  &  performance 
summary  screen 

■  To  facilitate  task  &  mission  execution  as  well  as  SA 

►  Reported  that  system  reduced  workload  by  making  info  more 
accessible 

■  Accessibility  helped  individuals  anticipate  increases  in  workload 

*  Liked  ability  to  display  info  concurrently 

*  Some  felt  number  of  screens  too  high  -  most  thought  four  about 
right 

*  Majority  hinted  at  information  load 

*  Common  problem  in  network-centric  warfare 

*  Some  thought  DSS  may  be  overly  complex 


Conclusions 
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•  Demonstration  outcomes  indicated  that  integration 
process  was  a  success 

■  A  viable  testbed  to  investigate  FORCEnet  concepts,  processes, 
&  technologies  is  at  hand 

«  Participants  embraced  the  available  information  and  technology 

■  Felt  information  &  technology  facilitated  compietion  of  tasks  and 
overali  mission 

»  Improvements 

•  Development  of  computer  based  agents  to  extend  user’s 
capabilities  and  control  overhead 

■  Deeper  integration  of  DDD,  DSS,  &  K-Desk  to  increase  types  of 
information  that  can  be  presented 

►  A  testbed  to  examine  organizational  designs  in  modern  military 
organizations  such  as  the  Expeditionary  Strike  Group,  based  on 
emerging  FORCEnet  concepts 


The  A2C2  Project 
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■  Objectives 

►  Explore  novel  command  &  control  organizations 

»  Design  C2  organizations  that  support  emerging 
FORCEnet  concepts 

■  Take  advantage  of  network  connectivity  to  conduct  new  types 
of  operations 

*  Provide  a  testbed  to  investigate  novel  organizations  & 
associated  displays  and  tools 


Theoretical 

Concept 


Experiment  & 
Data  Analysis 


